A study of reinforcement learning with knowledge sharing for distributed autonomous system

Reinforcement learning is one of effect,ive controller for autonomous robots. Became it does not need priori knowledge and hehaviom to complete given tasks are obtained automatically by repeating trial and error. However a large number of trials are required to realize complex tasks. So the task that can be obt.ained using t.he real robot is restricted to simple ones. Comidering these points, various methods that improve the learninx cost of reinforcement learning had been proposed. In the method that uses miori knowledxe. the methods lose the autonomy that is most import&k feature of reinforcement learning in applying it to the robots. In the Dyna-Q, that is oiie of simple and effective reinforcement learning architecture integrat.ing online planning, a model of environment is learned from real experience and by utilizing the model to learn, the learning time is decreased. In this architecture, the autonomy is held, however the model depends on the task, so acquired knowledge of environment can not be reused to other tasks. In the real world, human beings can learn various behaviors to complete complex tasks without priori knowledge of the tasks. We can try to realize the task in our image without moving our body. After the training in the image, hy trying to the real environment., we save time to learn. It means that we have model of environment and %re utilize t.he model to learn. We consider that the key ability t.hat makes the learning process faster is construction of environment model and utilization of it.. In this paper. we have proposed a method to obtain an environment model that is independent of the task. And by utihing the model_ we have decreased learning time. We consider distributed autonomous agents; and we show that. the environment model is constructed quickly hy sharing the experience of each agent, even when each agent has own independent task. To demonstrate the effectiveness of the proposed method, we have applied the method to the Q-learning and simulations of a puddle world are carried out. As a result effective hehaviors have been obtained quickly. 0-7803-7866-0/03/$17.00 82003 IEEE 1120 Mitsuo Takeshita Dept. of Systems Engineering Fncluty of Engineering Okayama University